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Introduction 


The  objective  of  creating  a  computer  simulated  virtual  environment  for  medical  training  is  to 
provide  a  level  of  training  not  possible  using  traditional  methods.  Traditional  techniques 
for  teaching  trauma  procedures  have  depended  largely  on  the  existence  of  a  sufficiently 
large  number  of  proctors  with  adequate  surgical  skills  to  teach  trauma  procedures.  Other 
approaches  include  practice  on  animals,  but  animal  models  of  injury  often  do  not  reflect 
human  trauma,  and  raise  a  host  of  ethical  issues  concerning  procuring  and  maintaining 
animals  for  surgical  training.  Also,  practice  on  humans  and  animals  precludes  the  ability  to 
repeatedly  rehearse  specific  components  of  the  procedure  that  may  prove  challenging  or 
require  finely  tuned  motor  skills.  An  additional  concern  is  that  Department  of  Defense 
(DoD)  hospitals  are  usually  not  regional  trauma  centers,  so  that  physicians  and  allied  health 
personnel  in  the  military  may  not  obtain  significant  exposure  to  human  trauma  cases  for 
training  purposes. 

Although  the  airline  industry  and  DoD  have  used  flight  and  battlefield  simulators  for  many 
years,  several  technical  challenges  have  limited  the  use  of  computer-based  simulation 
technology  in  medical  education.  A  flight  simulator  is  easier  to  implement  than  a  surgical 
simulator;  the  terrain  of  the  ground  is  fixed  and  rigid,  and  an  airplane  simply  moves 
through  a  path  above  this  terrain.  A  surgical  simulator,  on  the  other  hand,  is  more 
complex.  The  terrain  of  the  body  (the  internal  organs)  must  be  interacted  with,  and  they 
must  flex,  be  able  to  be  cut  and  then  re-attached.  This  manipulation  involves  much  greater 
computational  sophistication.  The  organs  in  the  body  must  be  programmed  with 
physiological  behaviors  and  basic  principles  of  physics  so  they  respond  appropriately  when 
they  are  cut,  tugged,  and  stretched.  In  addition,  the  surgic^  simulator  must  have 
knowledge  of  how  each  instrument  interacts  with  the  tissues.  For  example,  a  scalpel  will 
cut  tissue  when  a  certain  amount  of  pressure  is  applied;  however,  a  blunt  instrument  may 
not-  this  fact  must  be  simulated. 

Simulation  of  open  surgery  has  always  been  one  of  the  most  challenging  problems  for 
anyone  developing  surgery  simulation  system.  Unfortunately,  it  has  remained  an  elusive 
goal,  largely  considered  impossible  with  the  current  technology.  The  biggest  challenge,  of 
course,  is  the  fact  that  surgeons  use  their  hands  extensively  during  surgery  to  directly 
manipulate  organs  and  tissue.  The  human  hand  is  an  incredibly  sensitive  and  versatile 
instrument.  To  develop  a  virtual  environment  which  would  react  realistically  to  the  actions 
of  a  hand,  and  at  the  same  time  provide  the  required  degree  of  haptic  stimulation,  is  simply 
impossible  using  today’s  technology.  But  just  how  far  have  we  actually  traveled  towards 
that  elusive  goal?  We  will  try  to  answer  these  questions  in  this  paper  by  describing  the  HT 
Abdominal  Trauma  Simulator  (HATS)  test-bed.  This  simulator  has  been  developed  for 
open  surgery  from  the  front  to  remove  a  kidney  that  has  been  shattered  as  the  result  of  a 
blunt  trauma  to  the  body.  At  some  time  during  the  project  it  became  clear  that  it  would 
never  be  possible  to  model  realistically  the  complex  interaction  between  a  surgeon’s  hands 
and  the  contents  of  the  abdominal  cavity.  The  only  way  it  would  be  possible  to  train  these 
steps  of  the  procedure  was  by  substituting  them  with  something  else  which  could  provide 
at  least  some  level  of  cognitive  training. 

The  Solution 

Virtual  reality  technology  holds  tremendous  promise  for  surgical  trauma  training,  because  it 
offers  physicians,  battlefield  and  emergency  medical  personnel  the  opportunity  to  practice 
in  an  environment  where  mistakes  do  not  adversely  affect  patients.  An  optimal  training 
simulator  accurately  replicates  the  physical  and  physiological  properties  of  the  red 
procedure.  In  addition,  it  offers  the  abihty  to  automatically  track  and  evaluate  performance. 
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provides  the  option  of  different  procedural  scenarios,  as  well  as  simulating  a  range  of 
surgical  complications  and  anatomical  anomalies. 

Computerized  surgical  simulations  will  make  a  tremendous  impact  in  improving  surgical 
morbidity  and  mortality.  Studies  have  shown  that,  for  a  wide  range  of  diagnostic  and 
therapeutic  procedures,  doctors  doing  their  first  few  to  several  dozen  cases  are  much  more 
likely  to  make  a  greater  number  of  errors.  Adequate  proctoring  of  learners  by  experienced 
surgeons  is  cumbersome,  as  there  are  few  surgeons  experienced  enough  in  the  techniques 
to  proctor  their  colleagues.  It  is  exceedingly  difficult  for  physicians,  particularly  those  in 
rural  areas,  to  travel  to  larger  medical  centers  for  training.  The  requirement  also  places  a 
burden  on  experts  who  could  become  overwhelmed  with  proctoring  requests. 

Because  the  anatomical  models  used  in  the  surgical  simulator  can  be  based  on  clinical 
studies  (CT  and  MR  scans),  future  versions  of  the  proposed  simulation  could  be 
individualized  on  a  patient  basis  —  to  allow  pre-surgical  planning  and  allow  physicians  to 
practice  difficult  operations.  Surgical  trauma  simulators  will  allow  physicians  to 
communicate  surgical  scenarios  with  their  peers  and  exchange  ideas  regarding  surgical 
management  of  patients.  Furthermore,  these  simulations  could  be  transported  to  rural  areas 
to  allow  the  dissemination  of  surgical  techniques.  Significant  operative  risk  reduction  will 
be  made  possible  by  the  development  of  a  simulator  which  would  allow  transference  of 
skills  from  the  simulation  to  the  actual  patient  contact. 

HATS  has  become  a  test-bed  for  the  implementation  of  a  range  of  different  simulation 
techniques  with  varying  levels  of  realism  and  immersion.  Some  of  these  can  be 
characterized  as  multimedia  style,  while  others  are  full-blown  haptics  and  physics 
simulations. 
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Experimental  Methods 

Most  of  the  early  work  on  simulators  for  surgery  in  the  abdomen  and  other  soft  objects  has 
been  characterized  by  the  application  of  one  particular  physical  model.  Cover  et  al.  [5] 
were  the  first  to  present  re^-time  models  for  surgery  simulation.  They  used  a  simple 
surface-based  mass-spring  model  to  simulate  deformation  of  a  gallbladder.  Kuhn, 
Kuhnapfel  et  al.  [8]  have  implemented  mass-spring  models  in  the  OSMET  simulation 
system.  Although  their  models  in  principle  are  surface  models,  they  introduce  volumetric 
behavior  by  including  parent  nodes  that  coimect  nodes  on  different  sides  of  an  object. 
Surface  models  are  dso  used  in  the  commercial  Teleos  software  [9]  developed  by  HT 
Medical,  Inc.  Teleos  uses  tubular  spline  surface  models  and  can  model  simple  structures 
derived  from  the  tubular  topology  (e.g.  arteries,  gall-bladder).  Implicitly  solved  finite 
element  systems  have  been  used  in  the  on-going  parallel  work  of  Bro-Nielsen  [1,3]  and 
Cotin  et  al.  [4].  These  models  present  a  better  and  faster  solution  to  the  deformation 
problem  than  mass-spring  models.  But  at  the  same  time  they  are  more  complex.  See  [3]  for 
a  discussion  of  finite  element  models  and  their  comparison  to  mass-spring  models. 

The  technology  for  deforming  organs  has  become  quite  well  understood.  There  are  still  a 
lot  to  learn,  but  the  available  models  can  solve  a  range  of  problems.  As  a  consequence, 
several  realistic  simulators  have  started  to  appear.  Most  authors  are  taking  a  more  systems¬ 
like  approach  to  surgical  simulation.  Sagar  et  al.  [10]  have  presented  an  eye  surgery 
simulator  using  non-linear  finite  element  models  to  model  deformation  of  an  eye.  This 
system  is  complete  with  force  feedback  and  allows  cutting  in  the  eye.  The  work  of  Kuhn, 
Kuhnapfel  et  [8],  which  we  mentioned  above,  also  represents  an  interesting  system  for 
laparascopic  gall-bladder  surgery. 

In  [7],  Gibson  et  al.  presented  some  early  results  of  their  effort  to  develop  an  arthroscopic 
knee  surgery  simulator.  This  simulator  uses  a  new  volumetric  approach  to  model  organs. 
Although  this  is  a  promising  technique  for  the  future,  computers  are  probably  still  too  slow 
to  allow  realistic  deformation  of  a  volumetric  representation.  Recently,  Wiet  et  al.  have 
presented  an  endoscopic  sinus  surgery  simulator  [11]  which  is  based  on  the  use  of 
volumetric  models  too.  A  characteristic  of  both  of  these  simulators  is  that  they  operate  in  a 
scene  that  is  mostly  rigid.  The  amount  of  deformation  is  limited  and  can  therefore  be 
handled  with  simple  and  fast  algorithms.  At  HT  Medical  we  have  developed  a  number  of 
simulator  systems  in  recent  years,  including  simulators  for  neuro-endoscopic  surgery, 
interventional  radiology,  rigid  bronchoscopy,  and  flexible  ureteroscopy.  [2]  contains  a 
description  of  these  simulators. 

The  research  completed  under  the  DARPA  grant  have  further  defined  the  state-of-the-art  in 
medical  simulation.  There  have  been  seven  primary  elements  in  the  HATS  research  effort. 
These  efforts  include  the  following: 

1.  Development  of  haptic  feedback  interface  device 

2.  Refinement  in  computer  modeling  technology 

3.  Acquisition  and  processing  of  polygonal/segmented  abdominal  anatomy  database 

4.  Development  of  efficient  patient  specific  visualization  algorithms 

5.  Development  of  fluid  flow  simulation  model 

6.  Development  of  educational  content 

7.  Medici  Simulation  Software  Architecture 

Each  of  these  elements  is  described  below  in  detail  with  a  description  of  the  research 
methods. 
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1.  Development  of  haptic  feedback  interface  device 

Haptic  fidelity  is  a  critical  elements  to  a  realistic  trauma  simulation.  Two  general 
approaches  have  been  pursued  in  the  development  of  the  HATS  test-bed.  One  approaeh, 
whieh  eventually  was  determined  to  show  the  most  promise  in  terms  of  adaptability  and 
fidelity,  is  the  use  of  tactile  feedback  robotics.  The  second  approach  that  was  researched  is 
a  hybrid  system  integrating  robotics,  position  tracking,  proxy  physical  models,  and 
advanced  graphics  registration  and  over-lay  techniques. 

Approach  1:  Tactile  Feedback  Robotics 

The  SenSable  Technologies  Phantom  haptic  interface  device  was  used  to  gather  3D 
positioning  input  and  provide  force-feedback.  This  device  provides  6  degrees  of  freedom 
for  position  and  3  degrees  of  free  for  force-feedback.  Instead  of  developing  the  basic 
force-feedback  software,  we  have  used  the  SenSable  Technologies  GHOST  software 
package.  GHOST  implements  basic  collision  detection  and  force-feedback  and  we  have 
used  it  as  the  basie  component  in  the  system  for  colhsion  detection  and  force-feedback. 
Around  it  we  had  added  intelligent  control  of  models,  model  complexity,  and  force- 
feedback  characteristics  for  deformable  models.  The  final  configuration  of  HATS  used  the 
Phantom  in  conjuction  with  the  CRT  display  device.  This  combination  demonstrated  the 
greated  fidelity  and  adaptability  to  a  wide  variety  of  anatomic  variation  and  trauma 
pathology. 


Figure  1.  General  setup  of  the  HATS  simulation  system. 

A  significant  number  of  additional  position  tracking  and  force  feedback  devices  were  tested 
prior  to  focusing  on  the  Phantom  system.  These  devices  included  the  following; 


Cybernet  Systems  six  dof  force  feedback  system 
Polhemus  tracking  system 
Microsoft  force  feedback  joystick 
Immersion  six  dof  tracking  probe 
Immersion  four  dof  force  feedback  system 
Inunersion  laparoscopic  tracking  system 
Exos/Mareus  force  feedback  technologies 
Bertec  co-axial  tracking  and  feedback  device 
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CH  Products  force  feedback  joystick 

Sensable  Technologies  Phantoms  version  I  and  version  II  with  gimble 

Evaluation  criterea  included  tactile  resolution,  communication  protocol  issues,  dof  cability 
versus  HATS  requirements,  and  expandability.  The  Sensible  Technology  Phantom  device 
proved  to  have  the  higest  ratings  relative  to  HATS  requirements. 

Approach  2:  Hybrid  Tactile  Feedback  System 

The  system  used  a  blue-painted  physical  model  of  abdominal  organs  registered  with 
graphics  texture  maps  derived  from  corresponding  anatomical  models  derived  from  the 
Visible  Human  database.  The  user  would  wear  a  head-mounted  display  (HMD)  outfitted 
with  lipstick-size  video  cameras  and  tracking  devices.  The  physical  organs  contained 
sensors  that  were  tracked  by  the  HMD,  and  the  user  was  able  to  feel  the  organs  and  see  his 
hands  interact  with  visually  accurate  anatomy. 

The  research  and  development  of  the  Abdominal  Wound  Simulator'  exploited  advanced 
technology  in  several  areas: 

•  Computer  graphics  -  models,  programming 

•  Life-like  physical  models  (materi^s  science,  special  effects) 

•  Matte  techniques  -  'chroma  key',  'blue  screen'  compositing 

•  Motion  tracking 

•  Flexible  real-time  software  architecture 

This  novel  approach,  used  'augmented  reality'  techniques  borrowed  from  the  motion 
picture  specM  effects  industry,  including  blue  screen',  performance  compositing  and 
motion  tracking  methods. 

The  technology  challenge  that  required  the  greatest  initial  research  was  in  the  area  of 
cinematographic  special  effects.  The  following  section  provides  a  description  of  'blue 
screen'  matte  methods  that  were  essential  for  execution  of  the  'Abdominal  Trauma 
Simulator'. 

Blue  Screen  Imaging:  For  most  applications,  creating  a  blue  screen  composite  image  starts 
with  a  subject  that  has  been  photographed  in  front  of  an  evenly  lit,  bright,  pure  blue 
background.  The  compositing  process,  whether  photographic  or  electronic,  replaces  all  the 
blue  in  the  picture  with  another  image,  known  as  the  background  plate.  Blue  screen 
composites  can  be  made  optically  for  still  photos  or  movies,  electronic^ly  for  live  video, 
and  digitally  to  computer  images.  Until  very  recently  all  blue  screen  compositing  for  films 
was  done  optically  and  all  television  composites  were  done  using  analog  real  time  circuits. 

Another  term  for  Blue  Screen  is  Chroma-Key.  Chroma-Key  is  a  television  process  only.  A 
more  sophisticated  television  process  is  Ultimatte,  also  the  name  of  the  company  that 
manufactures  Ultimatte  equipment.  Ultimatte  has  been  the  ultimate  in  video  compositing  for 
many  years.  With  an  Ultimatte  unit  it  is  possible  to  create  composites  that  include  smoke, 
transparent  objects,  different  shades  of  blue,  and  shadows.  Ultimatte  now  makes  software 
that  works  with  other  programs  to  create  digital  mattes,  called  Cinefusion. 

Chroma  Key:  The  Chroma  Key  process  is  based  on  the  luminance  key.  In  a  luminance  key, 
everything  in  the  image  over  (or  under)  a  set  brightness  level  is  "keyed"  out  and  replaced 
by  either  another  image,  or  a  color  from  a  color  generator.  Luminance  keying  works  great 
with  title  graphics,  but  not  so  great  for  making  live  action  composites.  When  people  need  to 
be  keyed  over  a  background  image,  problems  arise  because  people  and  their  clothing  have  a 
wide  range  of  tones.  Hair,  shoes  and  shadow  areas  may  1^  very  dark,  while  eyes,  skin 
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highlights  and  shirt  collars  can  approach  100%  white.  Those  areas  might  key  through  along 
with  the  background. 

Chroma  Key  creates  keys  on  just  one  color  channel.  Broadcast  cameras  use  three 
independent  sensors,  one  for  each  color,  Red,  Green  and  Blue.  Most  cameras  can  output 
these  RGB  signals  separately  from  the  Composite  video  signal.  So  the  original  chroma  key 
was  probably  created  by  feeding  the  blue  channel  of  a  camera  into  a  keyer.  This  works, 
sort  of,  but  soon  manufacturers  created  dedicated  chromakeyers  that  could  accept  all  3 
colors,  plus  the  background  composite  signal  and  the  foreground  composite  signal.  This 
made  it  possible  to  select  any  color  for  the  key  and  fine  tune  the  selection  of  the  color.  As 
keyers  became  more  sophisticated  with  finer  control  of  the  transition  between  background 
and  foreground,  the  effect  became  less  obvious  and  jarring.  Today's  high-end  keyers  can 
make  a  soft  key  that  is  basically  invisible. 

The  reason  for  using  the  color  blue:  Red,  green  and  blue  channels  have  all  been  used,  but 
blue  has  been  favored  for  several  reasons.  Blue  is  the  complementary  color  to  flesh  tone- 
since  the  most  common  color  in  most  scenes  is  flesh  tone,  the  opposite  color  is  the  logical 
choice  to  avoid  conflicts.  Historically,  cameras  and  film  have  been  most  sensitive  to  blue 
light,  although  this  is  less  true  today.  Sometimes  (usually)  the  background  color  reflects 
onto  the  foreground  talent  creating  a  slight  blue  tinge  around  the  edges.  This  is  known  as 
blue  spill. 

Paints  and  Backings:  The  standard  paints  which  almost  everyone  uses  are  from  Rosco,  the 
light  gel  manufacturer.  They  make  ChromaKey  Blue  and  Green,  as  well  as  Ultimatte  Blue 
and  Green. 

Ultimatte:  Ultimatte  is  a  trademark  of  the  Ultimatte  Corporation,  of  Chatsworth  CA.  It  is  an 
outgrowth  of  work  the  company's  founder,  Petro  Vlahos,  did  in  the  1960s  for  the  Motion 
Picture  Research  Council.  The  goal  was  to  invent  a  better  matting  system  for  motion 
pictures.  Electronic  technology  was  not  ready  yet  then  for  a  film  resolution  system,  but 
video  could  be  achieved,  and  so  the  first  Ultimatte  units  were  created  in  the  70's. 

It  is  useful  to  think  of  the  Ultimatte  process  as  a  mixing  process,  not  a  keying  process.  This 
is  why  it  is  possible  to  matte  with  shadows,  hair,  water  etc.  An  Ultimatte  uses  the  intensity 
and  purity  of  the  blue  signal  as  a  function  to  determine  how  much  blending  to  perform 
between  the  foreground  and  background  images.  Another  useful  feature  of  the  Ultimatte  is 
the  previously  mentioned  blue  spill  removal.  Other  circuits  deal  with  glare,  uneven  or  dirty 
blue  backings,  etc.  Modem  units  from  the  Model  V  and  up  can  independently  adjust  the 
color  of  the  background  and  foreground  plates.  An  Ultimatte  used  to  have  many  knobs  on 
its  front  panel,  but  the  new  digital  units  use  a  display  screen  and  multifunction  controls.  A 
very  useful  feature  is  Screen  Correction,  which  allows  the  operator  to  create  perfect  mattes 
from  really  bad  blue  backings.  Ultimattes  can  retain  shadows  onto  the  background  plate. 

Research  into  the  interface  device  has  included  solicitation  for  input  from  numerous 
orgaitizations  with  diverse  experience.  These  organizations  included  the  following: 

Physical  models: 

•  Pink  House  Studios,  Inc. 

•  Fiber-Resin  Coip  (Full-Flesh™) 

•  Burman  Industries 

•  Alcone/Paramount  Company  Inc. 

•  Mike  Maddi  (Saturday  Night  Live  -  make-up  artist) 

•  Polytek  Dept.  FX 
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Animatronics,  mechanical  FX: 

•  Animax  Designs  Inc. 

•  EFFECTive  ENGINEERING 

3D  Digitizing/Compositing: 

•  Cyber  F/X 

•  Digital  Imaging  Systems  Inc. 

•  Photron 

•  Rhythm  &  Hues 

Motion  Capture: 

•  Tsi 

•  Oxford  Metrics 

3D  Tracking: 

•  Polhemus 

•  Immersion  Technologies 

•  Ascension 

We  integrated  two  versions  of  SensABLE  Technologies  PHANToM  device  into  HATS. 
The  three  motors  of  the  PHANToM  are  able  to  replicate  high  fidelity  haptic  “images.”  The 
PHANToM  is  limited  in  it’s  DOF  for  active  feedback  and  based  on  these  limitations,  we 
focused  our  efforts  in  the  second  and  third  quarters  of  year  two  on  a  six-DOF  device  -  The 
Cybernet,  Inc.  robot.  Our  initial  demonstrations  of  the  graphic  overlay  techniques 
demonstrated  feasibility  of  the  graphics  overlay  technique  and  integration  of  a  six  degree- 
of-freedom  (DOF)  force  feedback  robot  with  a  custom  stylus. 

Significant  limitations  of  this  implementation  involved  the  adaptability  of  the  system  to 
represent  variations  in  anatomy,  pathology,  and  trauma.  In  addition  the  component  of  the 
system  that  involved  the  physic^  model  presented  a  challenge  regarding  maintenance  and 
therefor  limited  it’s  commercial  viability. 


2.  ReOnement  in  computer  modeling  technology 

For  the  deformation  of  surfaces,  we  have  used  mass-spring  systems  (simplified  FEM 
models,  see  [3]).  A  mass-spring  system  consists  of  a  number  of  vertices  with  masses,  and 
a  set  of  springs  connecting  these  vertices.  In  addition,  we  have  added  strut  springs  that 
anchor  the  mass-spring  surface  to  a  particular  default  shape  and  position.  When  the 
resulting  explicit  mathematical  problem  is  solved  in  reed-time,  the  surface  deforms  with  a 
physically-based  behavior  in  response  to  external  stimuli.  An  example  of  a  deforming 
stomach  is  shown  in  figure  3. 
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Figure  3.  Instrument  deforming  stomach. 


The  reason  why  a  more  complex  FEM  system  was  not  used  for  these  surface  models,  is 
that  they  had  to  be  cut  to  accommodate  incisions.  This  is  only  reasonable  with 
mathematically  explicit  models  such  as  mass-spring  models  (see  [3]  for  a  discussion). 
Cutting  is  accomplished  using  a  complex  set  of  basic  operations  applied  to  individu^ 
triangles.  Taking  into  account  the  range  of  different  cases  corresponding  to  cutting  from  an 
edge  to  another  edge,  a  comer  to  an  edge,  the  interior  of  a  triangle  to  an  edge,  etc.  yields  a 
dozen  different  cases.  In  each  of  these  cases  all  the  dependencies  in  the  general  model  have 
to  be  updated  which  is  not  a  trivial  task.  For  arteries  and  other  tubular  models  we  are  using 
a  simple  linear  mass-spring  model  as  the  spine  of  a  tubular  stmcture  based  on  connected 
contours.  Each  of  the  contours  are  linked  to  a  mass-vertex  on  the  spine,  and  thus  controlled 
by  it. 


3.  Acquisition  and  processing  of  polygonal/segmented  abdominal  anatomy 
database 

We  acquired  a  comprehensive  segmented  abdominal  dataset  of  the  Visible  Human  male 
through  a  subcontract  with  Visible  Productions,  LLC,  Ft.  Collins,  Colorado.  We  have 
eompleted  initial  processing  of  this  data  for  real-time  interaction.  This  processing  included 
decimation  of  the  models  to  minimize  the  polygon  count  and  rendering  as  solid  surface 
models.  On  the  SiUcon  Graphics  Infinite  Reality  system  we  achieved  frame  rates  of 
approximately  five  frames  per  second  on  the  complete  dataset  without  culling  or  other 
processing  techniques.  The  process  of  decimation  of  this  dataset  provided  to  be  laborious 
and  eventually  we  adopted  a  technique  using  the  original  dataset  from  the  National  Library 
of  Medicine  and  M-Vox  (Bro-Nielsen)  software  for  segmentation. 


4.  Development  of  efficient  patient  specific  visualization  algorithms 

The  availability  of  hardware  with  advanced  3D  graphics  features  opens  the  door  to  new 
kind  of  applications  and  provides  new  power  to  dgorithms  that  were  not  before  applicable 
in  interactive  applications.  One  of  the  most  interesting  feature  now  available  on  the  new 
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generation  Silicon  Graphics  workstations  is  the  ability  to  define  volumetric  textures.  A 
volumetric  texture  is  a  3D  region  where  texture  values  are  stored  in  a  three  dimensional 
grid.  Each  polygon  inside  the  volumetric  texture  assumes  the  texture  values  it  intersects. 
This  graphics  feature  provides  a  new  way  to  apply  interactive  algorithm  for  Volume 
Rendering  resulting  in  fast  rendering  updates.  Our  technology  allows  anatomical 
visualization  of  patient  specific  and  Visible  Human  volume  data  exphcitly  (rather  than 
through  a  mind's  eye  view). 

These  volume  visualization  algorithms  allow  display  and  analysis,  interactively  of 
volumetric  data  from  CT  or  MRI  scanners  and  full  color  data  set  (the  Visible  Human  data). 
The  software  runs  on  workstations  from  Silicon  Graphics  (High  and  Maximum  IMPACT) 
and  on  Onyx  Reality  Engine  and  the  new  Infinity  Reality.  Features  include  infinite  and 
interactive  point  of  view,  with  arbitrary  section  planes. 

It  is  anticipated  that  the  technology  will  integrate  with  haptic  input  devices,  allowing  the 
user  to  feel  the  data  as  a  real  object.  Feasibility  has  been  established  in  the  identification  of 
approximate  tissue  density,  as  derived  from  gray  scale  and  color  data  and  assignment  of 
tactical  feeling  to  volume  data.  This  will  open  the  door  on  surgical  simulation  directly  on 
patient  data  without  any  kind  of  computational  expensive  transformations.  Research  is 
continuing  to  allow  interactive  volume  deformation  and  cutting. 

Capabilities  include  the  following: 

•  Full  support  of  gray-scales  (MR/CT)  and  color  data. 

•  Resolution  up  to  256x256x384  voxel  in  interactive  frame  rate 

•  Interactive  manipulation  of  point  of  view 

•  Interactive  advanced  look-up  table  manipulation  to  select  different  kinds  of  tissues 

•  Arbitrary  clipping  plane  to  inspect  internal  structures 

•  Interactive  scaUng  of  models  and  brick  explosions 

•  Depth  resolution  ranging  from  1  to  1,000  polygon  planes 

This  technology  has  been  incorporated  into  a  tool  set  for  the  medical  visualization  industry 
in  a  product  called  TELEOS  Voxel  Visualizer'^’^  (T-Vox™).  Much  of  the  algorithmic 
research  for  this  code  has  been  accomplished  through  this  DARPA  grant.  The  refinement 
of  this  software  as  a  product,  was  supported  by  HT  Medical’s  cooperative  research 
agreement  with  the  US  Department  of  Commerce’s  Advanced  Technology  Program. 

T-Vox  is  interactive  volume  visualization  software  for  real-time  rendering  of  any  2D 
sequential  medical  data.  T-Vox  supports  data  from  both  gray-scale  (MRI,  CT,  and  3-D 
ultrasound)  and  color  (Visible  Human  Data  Set  and  confocd  microscopy)  sources. 


T-Vox™  1.0,  the  industry’s  fastest  volume  rendering  software,  was  launched 
at  the  Medicine  Meets  Virtual  Reality  (MMVR)  conference 
in  San  Diego,  CA,  January  22,  1997. 
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The  software  also  has  an  expandable  plug-in  architecture  for  the  development  of  custom 
options.  Plug-ins  available  for  purchase  with  T-Vox  1.1  include:  T-Views  -  Generates 
animation  files  in  MPEG  or  QuickTime  formats.  Includes  unlimited  numbers  of  cameras 
and  animation  steps.  T-Explorer  -  Positions  the  camera  precisely  where  the  user  wants  it. 
Works  in  conjunction  with  T-Views  to  precisely  generate  a  key  frame  for  fly-through 
animation  and  virtual  endoscopy.  T-Reports  -  Generates  HTML  documents  to  be  shared  on 
an  intranet  or  the  Internet.  Perfect  for  communicating  results  and  generating  archives  of 
studies.  T-Analyze  -  Easily  imports  ANALYZE  data  files. 


5.  Development  of  fluid  flow  simulation  model 

Realistic  fluid  flow  has  been  a  significant  challenge  for  real-time  simulation.  Two  different 
techniques  were  developed  to  allow  for  intravascular  flow  and  flow  through  a  hemorrhage 
or  cut.  We  initially  developed  intravascular  fluid  flow  models  based  on  ramp  constrained 
random  flow  of  color  values.  Variations  of  the  ramp  constraints  were  used  to  control  the 
viscosity  of  the  fluid.  This  technique  has  proved  to  be  adequate  for  visualization,  but 
limited  in  its  abihty  to  have  physical  and  physiologic  impact  that  replicates  the  tme 
properties  of  the  liquid.  Our  current  methodology,  which  has  now  been  implemented  and 
incorporated  into  our  interventional  radiology  simulation  system,  is  based  on  computational 
nodes  of  fluid  automata.  This  technique  allows  for  realistic  models  of  diffusion  and 
pressure  to  be  incorporated  into  the  simulation  system  and  for  changes  in  intravascular  flow 
to  result  in  systemic  changes. 

We  have  implemented  a  bleeding  algorithm  that  uses  a  diffusion-style  algorithm  to  model 
the  flow  of  blood  on  the  surface  of  polygonal  models.  For  performance  and  complexity 
reasons  the  blood  is  stored  only  at  the  vertices  of  the  polygonal  surface.  This  limits  the 
precision  of  the  blood  movement  to  the  distance  between  polygon  vertices,  but  allows  the 
cutting  algorithm  to  be  used  on  the  bleeding  surface  as  well.  In  addition  to  modeling  the 
flow  of  the  blood,  the  algorithm  also  modifies  the  polygonal  surface  using  a  bump¬ 
mapping  approach  to  provide  the  illusion  of  blood  lying  on  top  of  the  polygonal  model.  At 
rendering  time  a  vertex  with  blood  on  it  is  moved  in  the  normal  direction  of  the  surface. 
The  distance  is  determined  based  on  the  amount  of  blood  on  the  vertex. 


6.  Development  of  educational  content 

A  set  of  storyboards  was  developed  that  showed  the  surgical  approach  in  detailed  steps. 
These  storyboards  were  very  useful  both  in  terms  of  guiding  the  development  of  the 
technical  systems  for  the  simulator  and  also  as  the  basis  for  multi-media  guidance  during 
the  simulation.  Some  of  the  steps  in  the  procedure  are  difficult  or  impossible  to  model 
realistically.  Instead  we  had  to  substitute  some  of  the  training  steps  with  different  degrees 
of  non-immersive  simulation. 

A  general  description  of  the  procedure  is  described  below  (adapted  from  Taylor,  D.L.  and 
W.R.  Fair  (1985)) 

Major  indicator  of  serious  complications:  Loss  of  continuity  of  the  renal  capsule  and 
distraction  of  fragments. 

If  visualized  or  suspected,  major  perirenal  hematoma  indicates  the  necessity  for  opening  of 
the  retroperitoneum  and  exploration  of  the  kidney.  In  the  emergency  setting,  injudicious 
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exploration  may  lead  to  unnecessary  nephrectomy,  however,  several  studies  have  shown 
the  first  2-3  days  offer  the  best  window  of  opportunity  for  corrective  surgery. 

Surgical  approaches: 

(1)  Make  a  vertical  midline  incision  in  the  abdomen. 

(2)  Begin  exploration  of  the  abdomen,  following  establishment  of  hemostasis  -  include 
debridement  of  the  path  of  any  penetrating  object(s)  or  missile(s).  The  retroperitoneum  is 
approached  along  the  aorta  at  the  root  of  the  mesentery. 

(3)  If  you  find  the  existence  of  a  large  retroperitoneal  hematoma  or  other  indications  of  a 
major  renal  trauma,  open  the  retroperitoneum  medial  to  the  inferior  mesenteric  vein  and 
identify  the  aorta. 

Indications  for  surgical  exploration  of  major  trauma  (McAninch  and  Carroll  (1989)): 

•  Expanding  or  uncontained  hematoma 

•  Pulsatile  retroperitoneal  hematoma 

•  Major  urinary  extravasation 

•  Non- viable  renal  parenchyma  (loss  of  more  than  15%  of  the  kidney) 

•  Vascular  injury 

Please  note  that  some  of  these  indications  can  only  be  determined  using  pre-operative 
imaging  methods,  which  may  not  be  practical  in  a  battlefield  setting. 

(4)  Dissect  cephalad  along  the  anterior  surface  of  the  aorta  to  expose  the  left  renal  vein 
crossing  the  aorta. 

(5)  Vascular  control  is  obtained  by  clamping  the  renal  vein  and  artery  at  their  origins  from 
the  vena  cava  and  the  aorta,  (mistake  possible  here  -  if  the  surgeon  strays  too  far  cephalad 
and  lateral  to  the  aorta,  the  splenic  artery  and  vein  may  be  mistaken  for  the  left  renal  artery 
and  vein). 

(6)  Reflect  the  colon  prior  to  exploration  of  the  retroperitoneal  space. 

(7)  Use  the  principles  of  exploration  outlined  by  Scott  and  Carlton  and  their  colleagues 
(Scott,  Carlton,  and  Goldman  (1969);  Scott  and  Selzman  (1966): 

•  Obtain  early  vascular  control 

•  Debride  any  devitalized  tissue 

•  Obtain  hemostasis  by  suture  ligation 

•  Insure  watertight  closure  of  the  collecting  system 

•  Approximate  die  parenchymal  margins 

•  Obtain  extraperitoneal  drainage  of  the  renal  fossa 

(8)  Analysis/corrective  surgery  of  the  kidney: 

Substantive  damage  limited  to  a  portion  of  the  kidney  usually  indicates  partial  nephrectomy. 

Disruption  of  the  collecting  system  usually  occurs  through  damage  to  the  calyceal  fomices. 
This  repair  involves  watertight  suturing  of  the  collecting  system  at  the  level  of  the  fornix. 
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If  bilateral  renal  damage  is  present,  the  kidney  with  the  best  possibility  of  recovery  is 
repaired  first. 

Transcapsular  damage  is  often  present  in  major  renal  injuries,  especially  those  with 
penetrating  injuries  (eg,  bullets,  shrapnel;  see  Scott,  Carlton  and  Goldman  (1969)).  In 
these  cases: 

•  Ligate  all  major  vessels. 

•  Use  a  watertight  closure  for  the  collecting  system. 

•  Close  the  capsule  with  running  or  interrupted  sutures. 

•  Use  diverting  pyelostomy  for  clots  in  the  renal  pelvis. 

•  Make  sure  that  all  devitalized  tissue  is  debrided. 


7.  Medical  Simulation  Software  Architecture 

The  software  for  the  simulator  has  been  built  as  a  multimedia  system  with  VR  simulation. 
Patient  records  (GUI),  and  Training  instructions  and  guidance  (GUI)  (see  figure  2).  The 
latter  two  components  are  contained  in  a  Graphical  User  Interface  (GUI)  that  has  the 
appearance  of  die  standard  manila  folder  used  by  many  American  medical  institutions  to 
store  patient  information.  The  GUI  is  shown  in  figure  3. 


Figure  2.  Multimedia  framework 


Figure  3.  Graphical  User  Interface. 
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Because  of  the  computational  load  of  this  sort  of  simulation  system,  the  software  has  been 
developed  as  a  real-time  distributed  software  system  designed  to  mn  on  different  computers 
linked  with  an  ethemet.  The  software  contains  at  least  six  independent  software  processes 
that  take  care  of  different  components  of  the  simulation.  We  analyzed  the  communication 
between  the  different  processes  and  found  that  the  best  way  to  divide  the  processes  was  by 
putting  haptics  and  colhsion  detection  on  one  computer,  and  the  remaining  processes  on 
another  computer  -  both  having  shared  memory  between  their  respective  processes.  In 
practice  a  four  processor  SGI  ONYX-2  shared  memory  parallel  processing  computer 
served  as  the  main  platform  and  a  SGI  IMPACT  as  an  additional  compute  server.  To 
implement  the  communication  between  the  two  computers  we  used  the  Parallel  Virtual 
Machine  (PVM)  [6]  communication  protocol.  The  system  diagram  is  shown  in  figure  4. 


Figure  4.  Processes  on  the  two  computers. 


7.  Conclusion 

We  have  developed  a  surgical  simulation  test-bed  for  training  the  removal  of  a  shattered 
kidney  by  open  surgery  from  the  front  of  a  trauma  patient.  This  kind  of  surgery  is 
particularly  complex  and  difficult  to  simulate,  and  the  simulator  cannot  be  characterized  as 
being  fully  realistic.  With  current  computers  and  technology  it  is  not  possible  to  develop 
completely  life-like  simulators  for  open  surgery.  But  developing  the  test-bed  has  allowed 
us  to  study  the  interaction  of  a  number  of  different  technologies  for  deformable  models, 
cutting,  bleeding,  haptic  modeling,  interprocess  communication,  and  multimedia  interfaces. 
The  simulator  includes  some  of  the  newest  advances  in  deformable  models  and  related 
technologies.  Throughout  the  research  period  these  technologies  have  been  leveraged 
toward  the  development  of  other  simulators  at  HT  Medical  Systems.  The  technologies 
focused  on  six  areas  including: 

1 .  Development  of  haptic  feedback  interface  device 

2.  Refinement  in  computer  modeling  technology 

3.  Processing  of  a  large  deformable  polygonal  database 

4.  Development  of  efficient  patient  specific  visualization  algorithms 

5.  Development  of  fluid  flow  simulation  model 

6.  Medical  Simulation  Software  Architecture 

Each  of  these  areas  was  developed  with  a  goal  of  re-usability  toward  other  medical 
simulation  apphcations.  The  following  section  describes  the  results  from  each  technical 
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area  and  then  includes  examples  of  additional  applications  developed  by  HT  Medical  which 
have  leveraged  this  technology  development. 

1.  Development  of  haptic  feedback  interface  device 

The  link  between  the  human  sensory  and  motor  systems  and  the  computer  can  be  either  a 
barrier  to  effective  simulation  and  training  or  a  natural  conduit  through  which  information 
and  experiences  flow.  With  the  advent  of  realistic  simulations  and  computerized  training 
software  the  missing  link  has  been  the  ability  of  the  chnician  to  “feel”  and  physically 
interact  with  the  patient’s  anatomy,  both  externally  and  internally.  Although  research  in 
this  field  has  been  going  on  for  20  years  or  more,  developments  in  computer  chips,  motor 
technology  and  programming  interfaces,  even  though  greatly  spurred  by  other  markets 
such  as  the  interactive  entertainment  industry,  have  not  yet  made  tliis  technology  accurate  or 
inexpensive  enough  to  be  widely  useful  in  medical  simulation  and  training. 


HATS  with  Phantom  tactile  feedback  device 

Our  objective  with  this  interface  device  development  research  has  been  to  provide  realistic 
feedback  (force  and/or  otherwise)  to  a  physician  performing  surgery.  The  requirements  for 
the  specific  procedure  being  simulated  has  a  major  impact  on  the  specifications  for  the 
tactile  feedback  interface  device.  Open  surgery  simulation,  such  as  abdominal  trauma 
surgery  which  requires  direct,  between  the  fingertips,  interaction,  requires  a  level  of 
proprioceptive  sensation  that  is  beyond  the  current  state-of-the-art  in  haptic  technology. 
Procedures  involving  the  manipulation  of  an  instmment  such  as  a  scaple  is  possible  with 
the  use  of  exisiting  technologies  including  the  Sensible  Technologies,  Phantom. 

Lessons  learned  throughout  the  three  years  of  this  research  program  in  tactile  feedback  have 
been  intregrated  into  a  number  of  interface  devices  for  HT  Medical’s  simulation  products 
and  prototypes. 


A  class  of  nursing  students  with  HT  Medical’s 
CathSim™  simulator  with  AccuToucK^ 
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CathsimTM  represents  the  basis  of  HT  Medical’s  vascular  access  product  line.  It  consists  of 
the  AccuTouch™  tactile  feedback  interface  device,  and  software  modules  providing 
educational  content  for  catheter  insertion  procedures. 


The  CathSim™  I.V.  training  simulator  with 
AccuTouch™  tactile  feedback  interface  cfevice 

CathSim  is  a  Microsoft  Windows  compatible  product.  This  product  was  launched  in 
January  1998  targeting  nurse  training  institutions.  Additional  modules  are  under 
development  for  the  placement  of  a  centrd  venous  catheters,  PICC  lines,  and  other  catheter 
related  procedures. 


The  PreOp™  endoscopic  simulator  allows 
realistic  manipulation  of  a  proxy  flexible  endoscope 


2.  Reflnement  in  computer  modeling  technology 

Several  advances  in  modeling  technology  have  resulted  from  this  research.  Refinements  in 
collision  detection,  specifically  involving  deformable  objects  has  been  a  major  challenge  in 
medical  simulation.  Two  factors  have  facilitated  acheivment  of  our  goals  in  this  area; 
implementation  of  software  techniques  (fast-finite  elements,  spline-based  modeling,  and 
on-the-fly  polygonal  decimation  routines)  as  well  as  the  concurrent  and  ongoing 
improvements  in  graphics  accelerated  hardware  including  the  availability  in  1997  of  the 
Silicon  Graphics  Onyx  Infinite  Reality  workstation. 

Results  from  this  work  have  been  leveraged  across  several  of  HT  Medical’s  simulation 
systems.  HT  Medical  is  developing  a  Flexible  Bronchoscopy  simulator  with  the  partnership 
of  pulmonologists  and  pharmacology  experts  at  Merck  &  Co.  This  procedural  simulator 


Page  19 


allows  for  examination  of  the  primary,  secondary,  and  tertiary  branches  of  an  adult 
bronchus  modeled  from  the  Visible  Human  Project  data. 


HT Medical’s  pediatric  ridged  bronchoscopy  simulator  with  content  developed  in 
collaboration  with  Penn  State’s  Hershey  Medical  Center. 

HT  Medical  has  completed  a  Pediatric  Rigid  Bronchoscopy  Simulator  in  collaboration  with 
Penn  State's  Hershey  Medical  Center  (HMC).  Based  on  preliminary  quantitative 
evaluation,  HMC  concluded  that  virtual  reaUty  training  at  their  teaching  hospital  could 
tremendously  reduce  their  OR-based  training  time  and  costs.  HT  Medical's  bronchoscopy 
simulator  was  the  first  of  several  training  simulators  for  HMC's  "Virtual  Operating  Room." 
The  simulator  includes  case  history  and  associated  material  for  the  pediatric  case  and  allows 
use  of  a  range  of  actual  rigid  scopes  to  diagnose  subtle  abnormalities. 

In  addition,  HT  Medical  has  designed  a  virtual  Neuroendoscopy  simulator,  funded  in  part 
by  Johnson  «fe  Johnson  Professional  -  Codman,  for  exploration  of  the  ventricular  system  of 
the  central  nervous  system.  The  user  is  able  to  navigate  throughout  the  ventricles  using  an 
endoscopic  device  threaded  through  the  top  of  a  mannequin.  HT  Medical  has  also 
developed  simulations  in  the  areas  of  Ureteroscopy  and  Laparoscopy. 

3.  Processing  of  a  large  deformable  polygonal  database 

The  large  number  of  deformable  objects  within  the  environment  of  an  open  abdomen 
required  the  development  of  sever^  techniques  for  developmeing  and  manning  the 
database.  Intiatially  it  was  beheved  that  the  all  objects  within  the  interactive  environment 
would  be  active  throughout  the  complete  simulation.  The  processing  capability  of  the 
multiprocessor  graphics  supercomputers  still  required  the  segmentation  of  the  procedure 
into  component  parts  -  procedural  task  steps  -  that  could  be  active.  This  segementation 
solved  a  computational  bottle-neck  and  also  worked  within  an  empirically  effective 
educational  design. 
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HATS  stomach  after  initial  incision  has  been  performed 

Further  validation  of  the  large  database  manipulation  techniques  were  provided  through 
several  demonstration  projects  completed  in  collaboration  with  Glaxo  Wellcome  and  with 
Boehringer  Mannheim. 


Large  anatomical  databases  have  been  displayed  using  techniques  developed  for  the  HATS  research  and  have 
been  explored  by  thousands  of  physicians  at  several  international  conferences 

With  Glaxo  Wellcome,  HT  Medical  created  an  Upper  G.I.  Hy-Through  Game  called  VR 
Endoscopic  Challenge.  The  Endoscopic  Challenge  is  a  virtual  reality  'fly-through'  of  the 
esophagus.  Visitors  'pilot'  a  virtual  endoscope  and  must  target  erupting  acid  as  it  makes  its 
way  up  the  esophagus.  The  goal  is  to  reduce  acid  contact  time  and  to  relieve  heartburn  pain 
due  to  reflux  disease.  The  purpose  of  the  program  was  to  attract  physicians  to  the  Glaxo 
Wellcome  booth  during  Digestive  Disease  Week,  May  1996. 

Ischemic  Insult  -  At  the  1996  American  College  of  Cardiology  (ACC)  Conference,  HT 
Medical  debuted  a  virtual  reality  "fly-through"  simulation,  called  Ischemic  Insult,  involving 
the  heart  and  great  vessels.  This  simulation  focused  on  treatment  for  acute  myocardid 
infarction.  The  simulation  allows  doctors  to  experience  first-hand  how  to  open  up  blocked 
arteries  that  cause  heart  attacks.  Wearing  a  head  mounted  display  (n- vision),  physicians  feel 
as  if  they  are  being  miniaturized  and  injected  into  the  heart's  left  atrium.  Their  mission,  in 
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order  to  save  the  patient,  is  to  maneuver  through  the  chambers  of  the  heart  and  arteries,  find 
the  blood  clot,  and  destroy  it  -  in  three  minutes  or  less. 

4.  Development  of  efficient  patient  specific  visualization  algorithms 

Patient  specific  volume  rendering  at  interactive  frame  rates  is  now  possible  with  HT 
Medical’s  Teleos  Voxel  Visualization  (T-Vox™)  software.  This  software  is  now  in  use  by 
customers  at  many  of  the  world’s  leading  academic  and  medical  research  facilities. 

Capabilities  include  the  following: 

•  Full  support  of  gray-scales  (MR/CT)  and  color  data. 

•  Resolution  up  to  256x256x384  voxel  in  interactive  frame  rate 

•  Interactive  manipulation  of  point  of  view 

•  Interactive  advanced  look-up  table  manipulation  to  select  different  kinds  of  tissues 

•  Arbitrary  clipping  plane  to  inspect  internal  structures 

•  Interactive  scaling  of  models  and  brick  explosions 

•  Depth  resolution  ranging  from  1  to  1,000  polygon  planes 


Interactive  T-Vox^^  rendering  of  the  segmented 
skull  from  the  Visible  Human  Database 


5.  Development  of  fluid  flow  simulation  model 

Models  were  developed  for  both  intravascular  and  extravascular  fluid  flow.  Realism  in 
bleeding  never  fully  achieved  the  same  level  of  fidehty  as  other  elements  of  the  visual 
simulation.  The  core  methodology  of  particle-system  modeling  when  implemented  as  a 
particle  stream  or  as  the  basis  for  an  isosurface  rendering  is  hmited  in  either  the  number  of 
particles  that  can  be  rendered  in  real-time  in  the  first  case  or  the  computational  expense  of 
the  latter.  Some  representations  of  bleeding  with  the  use  of  texture  maps  and  clipping 
planes  provides  a  high  degree  of  visud  realism  for  certain  types  of  bleeding. 
Implementation  of  appropriatly  creative  graphics  “tricks”  is  still  needed  in  many  specific 
applications  of  extravascular  flow. 

We  have  acheived  a  very  high  level  of  fidehty  in  simulation  of  endovascular  flow.  The 
techniques  developed  have  been  integrated  into  interventional  flouroscopy  simulations  with 
striking  realism. 


Page  22 


HT  Medical’s  simulation  of  endovascular  flow  of  contrast  media  is 
difficult  to  differentiate  from  actual  flouroscopic  imaging  of  live  patients. 

HT  Medical  is  developing  the  PreOp  Endovascular  Simulator  to  train  clinicians  in 
procedures  such  as  balloon  angioplasty  and  stent  placement.  This  system,  which 
duplicates  the  look  and  feel  of  the  actual  procedure,  enables  clinicians  to  practice 
procedures  as  many  times  as  necessary  before  performing  the  procedure  on  a  patient. 

The  simulation  also  integrates  pharmaceuticals  and  devices  used  in  the  actual  procedure 
-  such  as  thrombolytic  agents,  contrast  media,  catheters,  sheaths,  guide  wires,  and 
stents.  The  simulator  incorporates  tactile  feedback  so  that  clinicians  manipulating  these 
devices  can  "feel"  sensations  experienced  during  procedures,  such  as  encountering  an 
unexpected  obstruction  in  the  artery. 

HT  Medical  unveiled  the  prototype  for  this  system  at  the  1996  meeting  of  the  Society  of 
Cardiovascular  and  Interventional  Radiology  (SCVIR)  and  is  currently  preparing  the 
system  for  routine  use. 

6.  Medical  Simulation  Software  Architecture 

The  software  architecture  issues  for  medical  simulation  center  on  the  multiple  time-critical 
computational  tasks  including  processing  of  graphics,  physics,  multimedia  elements,  and 
tactile  feedback.  The  complexity  of  HATS  necessitated  the  development  and 
implementation  of  a  comunications  protocol  called  Parallel  Virtual  Machine  (PVM).  PVM 
segmented  the  computational  tasks  into  collision  detection  and  force  feedback  on  a  single 
processor  SGI  Impact  computer  while  the  physics,  graphics,  and  communications  tasks 
were  distributed  on  a  four  processor  SGI  Onyx2  system. 

PVM  allows  the  multi-thread  computational  processes  to  occur  while  maintaining  real-time 
interactivity.  Lessons  learned  during  the  implementation  of  PVM  have  been  integrated  by 
HT  Medici  in  several  other  medical  simulation  systems.  HT  Medical’s  CathSim™  system 
allows  realistic  feel  of  needle  insertion  while  graphics  performance  remains  at  near  thirty 
frames  per  second  on  an  off-the-shelf  Windows-based  Pentium  II  single  processor 
computer. 

Discussion 

Cost  containment  programs  are,  and  will  continue  to  be,  an  inte^al  part  of  health  care 
delivery  systems  throughout  the  world.  At  the  same  time  there  is  an  absolute  need  to 
maintain,  and  in  some  areas  improve,  the  quality  of  care  that  can  be  provided. 


Current  training  relies  too  much  on  patient  contact,  resulting  in  high  costs  to 
the  healthcare  system  and  less  than  ideal  clinical  outcomes 

The  technologies  have  now  evolved  to  the  point  where  medical  simulation  and  imaging  is 
both  credible  and  cost-effective  for  training,  objective  evaluation  of  practitioner 
competence,  and  patient-specific  interventional  planning.  Within  the  healthcare  industry  it 
is  well  accepted  that  simulation  technology  will  be  a  significant  addition  to  current  pre¬ 
operative  planning  standards. 


Simulators  improve  training,  reduce  costs,  and  limit  patient  risk 


Many  medical  procedures  are  especially  suited  to  computer-based  simulation  for  training 
and  certification,  because  they  ^eady  place  the  physician  remotely  from  the  site  of  the 
operative  manipulation.  Current  medical  procedures  dependent  upon  remote  manipulation 
of  medical  devices  and  viewing  from  video  monitors  provide  an  easy  transition  to  the 
simulation  environment.  We  can  use  the  historical  experience  and  commercial  success  of 
flight  simulation  as  a  paradigm,  yet  the  physician  population  represents  more  than  ten  times 
the  number  of  commercial  airline  pilots.  Based  on  the  number  of  physicians  and  the 
millions  of  other  allied  healthcare  providers,  medical  simulation  represents  a  significant 
market  opportunity. 

Medical  simulators  allow  healthcare  providers  to  practice  procedures  in  an  environment 
where  mistakes  do  not  have  dire  consequences,  lowering  risk  associated  with  training  on 
human  patients,  avoiding  the  use  of  animals  for  training,  and  establishing  standards  and 
optimization  of  specific  procedures. 

The  availability  of  simulation  products  will  have  a  significant  effect  in  medical  cost 
reduction,  reduced  reliance  on  animals,  and  most  importantly,  the  availability  of  an 


effective  training  tool  will  signficantly  aid  in  appropriate  utilization  of  medical  devices  and 
will  improve  clinical  outcomes. 

Specific  examples  are  as  follows: 

Reduces  Complications 

Improved  training  will  result  in  fewer  complications.  Just  as  the  LINK  simulators 
in  the  1940s  reduced  fatal  crashes  during  night  aircraft  carrier  landings  by  95% 
during  World  War  H,  medical  simulators  will  reduce  errors  and  save  significant 
costs  associated  with  those  complications.  Simulation  could  bridge  the  gap  between 
the  number  of  actual  cases  performed  by  a  practitioner  and  the  total  number  of  cases 
needed  to  maintain  skills. 

Example:  A  study  from  Duke  University  (Jollis,  J.G.  1996)  showed  an  increase  of 
significant  complications  of  2  per  100  coronary  angioplasty  procedures  between 
high  and  low  volume  clinicians.  There  are  approximately  1,300  interventional 
cardiology  clinics  in  the  U.  S..  A  typical  institution  with  four  cardiologists 
performs  160  angioplasties  per  year.  Best  results  are  obtained  by  individuals 
performing  at  least  150  each.  Complications  can  result  in  the  need  for  open  surgery 
at  a  cost  of  $30,000  per  patient.  Saving  potential  is  160  x  0.02  x  $30,000  = 
$96,000  per  year  per  clinic  or  $125  million  nationally  for  angioplasty. 

Litigation 

Simulation  could  provide  evidence  that  adequate  training  is  provided  within  an 
institution  and  using  the  simulator  as  a  scoring  device  allows  hedthcare  providers  to 
demonstrate  competency.  In  December  of  1997  a  Connecticut  jury  awarded  $12.2 
million  to  a  physician,  who  as  a  resident  at  Yale  Medical  School,  was  infected  by 
AIDS  due  to  inadaquate  training  in  catheter  placement  technique  (New  York  Times, 
December  21,  1997).  Use  of  HT  Medical’s  CathSim™  Intravenous  Training 
System  may  have  prevented  this  injury  and  subsequent  liability. 

Increase  Utilization  of  New  Medical  Procedures 

The  availability  of  in-house  training  and  skills  maintenance  enhances  comfort  with 
new  medical  products  and  increases  utilization  by  a  conservative  estimate  of  20%. 
As  many  as  70%  of  graduates  from  animal  and  cadaver  courses  do  not  adopt  the 
procedures  in  their  practice  because  they  have  not  been  comfortable  with  the  amount 
and/or  quality  of  training. 

Reduced  Operating  Room  Time 

With  resident  training  the  learning  curve  would  be  accelerated  through  simulation. 

Example:  The  American  Heart  Association  (AHA)  guidelines  for  coronary 
angioplasty  state  that  residents  require  200  procedures  to  be  at  the  top  of  the 
learning  curve.  Training  adds  approximately  30  minutes  to  each  procedure.  The 
cost  of  operating  room  time  is  typically  $1,500  per  hour.  Saving  potential  is  200  x 
0.5  X  $1,500  =  $150,000  per  resident  or  $3  billion  nationally. 

Reduced  Use  of  Animals 

Computer-based  simulation  technology  will  replace  and  augment  current  models 
with  the  additional  benefits  of  anatomic  accuracy  that  can  not  be  obtained  with 
animals;  pathology  and  dynamic  properties  not  possible  with  human  cadavers;  and 
without  the  consumption  of  expensive  medical  devices. 
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A  typical  training  program  for  new  imaged  guided  medical  procedures  requires  two 
days  and  between  .66  to  1.0  animal  per  physician.  Each  animal  costs  $2,500.  HT 
Medical  will  eliminate  the  need  for  animals  during  the  first  day  of  the  training 
program.  This  proposed  system  will  save  half  the  animals  currently  used. 

Reduced  Waste  of  Medical  Devices  for  Training 

Simulation  is  non-destructive  to  medical  devices.  A  typical  department  of  a 
teaching  hospital  has  two  interventional  radiology  fellows  per  year  doing  100 
procedures  involving  stent  deployment.  Stents  cost  upward  of  $1,000  each. 
Approximately  10%  of  stents  are  wasted  during  the  learning  curve.  There  are 
approximately  1,000  teaching  hospitals  in  the  U.  S.  Saving  potential  is  100  x  0.1  x 
$1,000  =  $10,000  per  year  per  hospital  or  $10  million  nationally. 

Improves  training 

Using  simulation-based  virtual  environments  for  medical  training,  it  is  possible  to 
teach  healthcare  providers  more  thoroughly  than  current  approaches.  The  use  of 
medical  simulators  has  precedence  in  the  training  of  commercial  and  militaiy  pilots 
where  the  quality  and  d^epth  of  training  has  improved  and  cost  decreased  through 
flight  simulation  trainers.  With  modem  aircraft,  flight  training  would  be  impossible 
without  simulators.  The  same  can  be  anticipated  for  future  medical  practices. 

Localizes  Training 

Simulation  systems  will  eliminate  many  costs  associated  with  the  traditional  training 
at  remote  sites  and  wUl  limit  revenue  losses  due  to  absence  from  the  medical  facihty 
during  training.  Hospitals  and  clinics  that  have  simulation  systems  will  have 
immediate  access  to  new  device  training. 

Standardizes  Training  and  Certification 

Simulations  will  generate  more  consistent  and  predictable  patient  outcomes  and  will 
lower  complication  rates.  Standardized  care  is  a  central  focus  in  the  managed  care 
environment.  Hospitals  and  HMOs  will  use  these  systems  to  objectively  measure 
cognitive  and,  for  the  first  time,  motor-skills.  There  is  pressure  from  organizations 
like  the  American  College  of  Cardiology,  the  American  College  of  Surgeons,  and 
insurance  providers  for  hospitals  to  use  simulators,  when  they  become  available. 
Simulations  will,  in  the  near  future,  be  used  to  establish  that  physicians  have  the 
requisite  skills,  prior  to  patient  contact. 

Provides  Skills  Maintenance 

Skills  maintenance  will  be  available  without  large  patient  volumes.  Patient  referrals 
to  larger  medical  facilities  will  be  reduced  -  saving  money  in  travel  and  time  from 
work  for  patient. 


Safety 

Ability  to  practice  fluoroscopy  in  an  environment  without  exposure  to  x-ray 
radiation  -  standard  training  involves  use  of  a  real  fluoroscope.  A  wide  variety  of 
medical  techniques  will  be  simulated  that  could  not  otherwise  be  safely  and  ethically 
tested  and  taught  without  simulation.  In  addition,  HT  Medical’s  systems  provide 
the  ability  to  rehearse  a  variety  of  procedures  without  risk  of  exposure  to  infection. 

These  large  financial  benefits  of  simulation  provide  an  opportunity  to  satisfy  a  commanding 
imperative  of  the  healthcare  industry  for  cost  containment  by  supplying  cost-effective 
virtual  reality  training  products. 
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